Profiling Booking

Dataset statistics

Number of variables	12
Number of observations	6362620
Missing cells	0
Missing cells (%)	0.0%
Duplicate rows	543
Duplicate rows (%)	< 0.1%
Total size in memory	497.6 MiB
Average record size in memory	82.0 B

Variable types

NUM	6
BOOL	4
CAT	2

Warnings

Dataset has 543 (< 0.1%) duplicate rows	Duplicates
`nameDest_char` is highly correlated with `type`	High correlation
`type` is highly correlated with `nameDest_char`	High correlation
`amount` is highly skewed (γ1 = 30.99394948)	Skewed
`FluxOrig` is highly skewed (γ1 = -24.63052048)	Skewed
`FluxDest` is highly skewed (γ1 = 32.91634067)	Skewed
`oldbalanceOrg` has 2102449 (33.0%) zeros	Zeros
`oldbalanceDest` has 2704388 (42.5%) zeros	Zeros
`FluxOrig` has 2089037 (32.8%) zeros	Zeros
`FluxDest` has 2317292 (36.4%) zeros	Zeros

Reproduction

Analysis started	2020-10-25 19:51:45.384619
Analysis finished	2020-10-25 19:57:45.182772
Duration	5 minutes and 59.8 seconds
Software version	pandas-profiling v2.9.0
Download configuration	config.yaml

step
Real number (ℝ_≥0)

Distinct	743
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	243.3972456
Minimum	1
Maximum	743
Zeros	0
Zeros (%)	0.0%
Memory size	48.5 MiB

Quantile statistics

Minimum	1
5-th percentile	16
Q1	156
median	239
Q3	335
95-th percentile	490
Maximum	743
Range	742
Interquartile range (IQR)	179

Descriptive statistics

Standard deviation	142.331971
Coefficient of variation (CV)	0.5847723161
Kurtosis	0.329070555
Mean	243.3972456
Median Absolute Deviation (MAD)	92
Skewness	0.3751768885
Sum	1548644183
Variance	20258.38998
Monotocity	Increasing

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
19	51352	0.8%
18	49579	0.8%
187	49083	0.8%
235	47491	0.7%
307	46968	0.7%
163	46352	0.7%
139	46054	0.7%
403	45155	0.7%
43	45060	0.7%
355	44787	0.7%
Other values (733)	5890739	92.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	2708	< 0.1%
2	1014	< 0.1%
3	552	< 0.1%
4	565	< 0.1%
5	665	< 0.1%

Value	Count	Frequency (%)
743	8	< 0.1%
742	14	< 0.1%
741	22	< 0.1%
740	6	< 0.1%
739	10	< 0.1%

type
Categorical

HIGH CORRELATION

Distinct	5
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	48.5 MiB

CASH_OUT	2237500
PAYMENT	2151495
CASH_IN	1399284
TRANSFER	532909
DEBIT	41432

Frequencies
Length

Value	Count	Frequency (%)
CASH_OUT	2237500	35.2%
PAYMENT	2151495	33.8%
CASH_IN	1399284	22.0%
TRANSFER	532909	8.4%
DEBIT	41432	0.7%

Frequencies of value counts

Unique

Unique	0 ?
Unique (%)	0.0%

Histogram of lengths of the category

Length

Max length	8
Median length	7
Mean length	7.422395963
Min length	5

amount
Real number (ℝ_≥0)

SKEWED

Distinct	5316900
Distinct (%)	83.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	179861.9035
Minimum	0
Maximum	92445516.64
Zeros	16
Zeros (%)	< 0.1%
Memory size	48.5 MiB

Quantile statistics

Minimum	0
5-th percentile	2224.0995
Q1	13389.57
median	74871.94
Q3	208721.4775
95-th percentile	518634.1965
Maximum	92445516.64
Range	92445516.64
Interquartile range (IQR)	195331.9075

Descriptive statistics

Standard deviation	603858.2315
Coefficient of variation (CV)	3.357343715
Kurtosis	1797.956705
Mean	179861.9035
Median Absolute Deviation (MAD)	68393.655
Skewness	30.99394948
Sum	1.144392945e+12
Variance	3.646447637e+11
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
10000000	3207	0.1%
10000	88	< 0.1%
5000	79	< 0.1%
15000	68	< 0.1%
500	65	< 0.1%
100000	42	< 0.1%
21500	37	< 0.1%
120000	29	< 0.1%
135000	20	< 0.1%
0	16	< 0.1%
Other values (5316890)	6358969	99.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	16	< 0.1%
0.01	1	< 0.1%
0.02	3	< 0.1%
0.03	2	< 0.1%
0.04	1	< 0.1%

Value	Count	Frequency (%)
92445516.64	1	< 0.1%
73823490.36	1	< 0.1%
71172480.42	1	< 0.1%
69886731.3	1	< 0.1%
69337316.27	1	< 0.1%

oldbalanceOrg
Real number (ℝ_≥0)

ZEROS

Distinct	1845844
Distinct (%)	29.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	833883.1041
Minimum	0
Maximum	59585040.37
Zeros	2102449
Zeros (%)	33.0%
Memory size	48.5 MiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	14208
Q3	107315.175
95-th percentile	5823702.278
Maximum	59585040.37
Range	59585040.37
Interquartile range (IQR)	107315.175

Descriptive statistics

Standard deviation	2888242.673
Coefficient of variation (CV)	3.46360618
Kurtosis	32.96487854
Mean	833883.1041
Median Absolute Deviation (MAD)	14208
Skewness	5.249136421
Sum	5.305681316e+12
Variance	8.341945738e+12
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	2102449	33.0%
184	918	< 0.1%
133	914	< 0.1%
195	912	< 0.1%
164	909	< 0.1%
181	908	< 0.1%
109	908	< 0.1%
157	902	< 0.1%
146	899	< 0.1%
136	898	< 0.1%
Other values (1845834)	4252003	66.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	2102449	33.0%
0.05	1	< 0.1%
0.18	1	< 0.1%
0.21	1	< 0.1%
0.44	1	< 0.1%

Value	Count	Frequency (%)
59585040.37	1	< 0.1%
57316255.05	1	< 0.1%
50399045.08	1	< 0.1%
49585040.37	1	< 0.1%
47316255.05	1	< 0.1%

oldbalanceDest
Real number (ℝ_≥0)

ZEROS

Distinct	3614697
Distinct (%)	56.8%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	1100701.667
Minimum	0
Maximum	356015889.4
Zeros	2704388
Zeros (%)	42.5%
Memory size	48.5 MiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	132705.665
Q3	943036.7075
95-th percentile	5147229.713
Maximum	356015889.4
Range	356015889.4
Interquartile range (IQR)	943036.7075

Descriptive statistics

Standard deviation	3399180.113
Coefficient of variation (CV)	3.088193846
Kurtosis	948.6741254
Mean	1100701.667
Median Absolute Deviation (MAD)	132705.665
Skewness	19.92175792
Sum	7.003346437e+12
Variance	1.155442544e+13
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	2704388	42.5%
10000000	615	< 0.1%
20000000	219	< 0.1%
30000000	86	< 0.1%
40000000	31	< 0.1%
102	21	< 0.1%
198	19	< 0.1%
160	18	< 0.1%
125	18	< 0.1%
132	18	< 0.1%
Other values (3614687)	3657187	57.5%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	2704388	42.5%
0.01	1	< 0.1%
0.03	1	< 0.1%
0.13	1	< 0.1%
0.33	1	< 0.1%

Value	Count	Frequency (%)
356015889.4	1	< 0.1%
355553416.3	1	< 0.1%
355381433.6	1	< 0.1%
355380483.5	1	< 0.1%
355185537.1	1	< 0.1%

isFraud
Boolean

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	48.5 MiB

0	6354407
1	8213

Common Values
Chart

Value	Count	Frequency (%)
0	6354407	99.9%
1	8213	0.1%

isFlaggedFraud
Boolean

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	48.5 MiB

0	6362604
1	16

Common Values
Chart

Value	Count	Frequency (%)
0	6362604	> 99.9%
1	16	< 0.1%

FluxOrig
Real number (ℝ)

SKEWED
ZEROS

Distinct	2962285
Distinct (%)	46.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	21230.5645
Minimum	-10000000
Maximum	1915267.9
Zeros	2089037
Zeros (%)	32.8%
Memory size	48.5 MiB

Quantile statistics

Minimum	-10000000
5-th percentile	-74619
Q1	-10150.435
median	0
Q3	0
95-th percentile	251627.927
Maximum	1915267.9
Range	11915267.9
Interquartile range (IQR)	10150.435

Descriptive statistics

Standard deviation	146643.2865
Coefficient of variation (CV)	6.907177925
Kurtosis	1509.981332
Mean	21230.5645
Median Absolute Deviation (MAD)	7273.94
Skewness	-24.63052048
Sum	1.350820143e+11
Variance	2.150425347e+10
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	2089037	32.8%
-184	737	< 0.1%
-181	732	< 0.1%
-186	731	< 0.1%
-195	728	< 0.1%
-157	728	< 0.1%
-111	726	< 0.1%
-146	726	< 0.1%
-136	722	< 0.1%
-164	722	< 0.1%
Other values (2962275)	4267031	67.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-10000000	3	< 0.1%
-10000000	5	< 0.1%
-10000000	262	< 0.1%
-10000000	12	< 0.1%
-10000000	2	< 0.1%

Value	Count	Frequency (%)
1915267.9	1	< 0.1%
1821279.53	1	< 0.1%
1782621.49	1	< 0.1%
1781905.25	1	< 0.1%
1675137.6	1	< 0.1%

FluxDest
Real number (ℝ)

SKEWED
ZEROS

Distinct	4011056
Distinct (%)	63.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	124294.7317
Minimum	-13060826.21
Maximum	105687838.8
Zeros	2317292
Zeros (%)	36.4%
Memory size	48.5 MiB

Quantile statistics

Minimum	-13060826.21
5-th percentile	-221270.2365
Q1	0
median	0
Q3	149105.425
95-th percentile	559989.539
Maximum	105687838.8
Range	118748665
Interquartile range (IQR)	149105.425

Descriptive statistics

Standard deviation	812939.0811
Coefficient of variation (CV)	6.540414626
Kurtosis	1765.310944
Mean	124294.7317
Median Absolute Deviation (MAD)	62705.85
Skewness	32.91634067
Sum	7.908401457e+11
Variance	6.608699495e+11
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	2317292	36.4%
10000000	136	< 0.1%
-10000	77	< 0.1%
-5000	75	< 0.1%
-15000	63	< 0.1%
-500	59	< 0.1%
-21500	35	< 0.1%
-100000	35	< 0.1%
120000	27	< 0.1%
-135000	17	< 0.1%
Other values (4011046)	4044804	63.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-13060826.21	1	< 0.1%
-9681485.46	1	< 0.1%
-6755513.96	1	< 0.1%
-6214174.81	1	< 0.1%
-5353303.67	1	< 0.1%

Value	Count	Frequency (%)
105687838.8	1	< 0.1%
100325160.3	1	< 0.1%
99045450.43	1	< 0.1%
92445516.64	1	< 0.1%
85923104.65	1	< 0.1%

amount_anomaly_orig
Boolean

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	6.1 MiB

True	3939445
False	2423175

Common Values
Chart

Value	Count	Frequency (%)
True	3939445	61.9%
False	2423175	38.1%

amount_anomaly_dest
Boolean

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	6.1 MiB

False	4924370
True	1438250

Common Values
Chart

Value	Count	Frequency (%)
False	4924370	77.4%
True	1438250	22.6%

nameDest_char
Categorical

HIGH CORRELATION

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	48.5 MiB

C	4211125
M	2151495

Frequencies
Length

Value	Count	Frequency (%)
C	4211125	66.2%
M	2151495	33.8%

Frequencies of value counts

Unique

Unique	0 ?
Unique (%)	0.0%

Histogram of lengths of the category

Length

Max length	1
Median length	1
Mean length	1
Min length	1

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Cramér's V (φc)

Cramér's V is an association measure for nominal random variables. The coefficient ranges from 0 to 1, with 0 indicating independence and 1 indicating perfect association. The empirical estimators used for Cramér's V have been proved to be biased, even for large samples. We use a bias-corrected measure that has been proposed by Bergsma in 2013 that can be found here.

Count
Matrix

First rows

	step	type	amount	oldbalanceOrg	oldbalanceDest	isFraud	FluxOrig	FluxDest	amount_anomaly_orig	amount_anomaly_dest	nameDest_char
0	1	PAYMENT	9839.64	170136.00	0.0	0	-9839.64	0.00	False	False	M
1	1	PAYMENT	1864.28	21249.00	0.0	0	-1864.28	0.00	False	False	M
2	1	TRANSFER	181.00	181.00	0.0	1	-181.00	0.00	False	True	C
3	1	CASH_OUT	181.00	181.00	21182.0	1	-181.00	-21182.00	False	True	C
4	1	PAYMENT	11668.14	41554.00	0.0	0	-11668.14	0.00	False	False	M
5	1	PAYMENT	7817.71	53860.00	0.0	0	-7817.71	0.00	False	False	M
6	1	PAYMENT	7107.77	183195.00	0.0	0	-7107.77	0.00	False	False	M
7	1	PAYMENT	7861.64	176087.23	0.0	0	-7861.64	0.00	False	False	M
8	1	PAYMENT	4024.36	2671.00	0.0	0	-2671.00	0.00	True	False	M
9	1	DEBIT	5337.77	41720.00	41898.0	0	-5337.77	-1549.21	False	True	C

Last rows

	step	type	amount	oldbalanceOrg	oldbalanceDest	isFraud	FluxOrig	FluxDest	amount_anomaly_orig	amount_anomaly_dest	nameDest_char
6362610	742	TRANSFER	63416.99	63416.99	0.00	1	-63416.99	0.00	False	True	C
6362611	742	CASH_OUT	63416.99	63416.99	276433.18	1	-63416.99	63416.99	False	False	C
6362612	743	TRANSFER	1258818.82	1258818.82	0.00	1	-1258818.82	0.00	False	True	C
6362613	743	CASH_OUT	1258818.82	1258818.82	503464.50	1	-1258818.82	1258818.83	False	True	C
6362614	743	TRANSFER	339682.13	339682.13	0.00	1	-339682.13	0.00	False	True	C
6362615	743	CASH_OUT	339682.13	339682.13	0.00	1	-339682.13	339682.13	False	False	C
6362616	743	TRANSFER	6311409.28	6311409.28	0.00	1	-6311409.28	0.00	False	True	C
6362617	743	CASH_OUT	6311409.28	6311409.28	68488.84	1	-6311409.28	6311409.27	False	True	C
6362618	743	TRANSFER	850002.52	850002.52	0.00	1	-850002.52	0.00	False	True	C
6362619	743	CASH_OUT	850002.52	850002.52	6510099.11	1	-850002.52	850002.52	False	False	C

Most frequent

	step	type	amount	oldbalanceOrg	isFraud	FluxOrig	FluxDest	amount_anomaly_orig	amount_anomaly_dest	nameDest_char	count
498	387	CASH_OUT	10000000.00	10000000.0	1	-10000000.0	10000000.0	False	False	C	4
532	617	CASH_OUT	10000000.00	10000000.0	1	-10000000.0	10000000.0	False	False	C	4
534	646	CASH_OUT	10000000.00	10000000.0	1	-10000000.0	10000000.0	False	False	C	3
0	7	PAYMENT	1849.50	0.0	0	0.0	0.0	True	False	M	2
1	8	PAYMENT	7759.31	0.0	0	0.0	0.0	True	False	M	2
2	9	PAYMENT	2388.93	0.0	0	0.0	0.0	True	False	M	2
3	9	PAYMENT	2783.83	0.0	0	0.0	0.0	True	False	M	2
4	9	PAYMENT	6499.28	0.0	0	0.0	0.0	True	False	M	2
5	9	PAYMENT	6755.57	0.0	0	0.0	0.0	True	False	M	2
6	9	PAYMENT	10042.85	0.0	0	0.0	0.0	True	False	M	2

Overview

Variables

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Cramér's V (φc)

Missing values

Sample

First rows

Last rows

Duplicate rows

Most frequent